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Coaching and the SAT® I 


E ach year well over one million students take 
the SAT® I as they prepare for the transition 
from high school to college. In hopes of 
increasing scores, many of these students and their 
parents struggle with the decision about whether to 
invest substantial amounts of time and money in 
commercial coaching programs. 

Students and parents often hear anecdotal 
accounts of large score increases from friends, 
neighbors, or co-workers. Yet such individual exam- 
ples do not provide a realistic indication of the 
average or typical effects of commercial coaching or 
of how another student might fare as a result of such 
coaching. Further complicating the decision are the 
claims of large score gains (and even guarantees) by 
some commercial coaching firms. On closer inspec- 
tion, the reportedly high gains are computed using 
flawed research methods, and the "guarantees" may 
simply permit students to continue participating in 
coaching programs at no additional charge after 
initially paying a large fee. 

To complicate matters, there has been little 
objective information about the possible benefits of 
coaching programs to help guide decisions by 
parents and students. Recently, however, a study 
sponsored by the College Board reexamined the 
effects of coaching on the SAT I (Powers and Rock, 
1998). The investigators documented the test prepa- 
ration strategies of a large sample of test takers. Then 
they contrasted the typical or average score gain on 
the test following participation in formal coaching 
programs (i.e., programs external to the high school 
and generally conducted by commercial companies) 
with the score gains of students not participating in 

such programs. This 
paper describes both 
the results of this study 
and selected previous 
research on coaching for 
the SAT I. (The SAT I, an 


enhanced revision of the former SAT, was introduced 
at the March 1, 1994 administration. Coaching 
studies that were conducted before that date refer to 
the original SAT and are so identified.) 

COACHING PROGRAMS 

Coaching programs differ in terms of methods, dura- 
tion, and objectives. They may focus differentially on 
various elements (Allalouf & Ben-Shakhar, 1998): 

(1) familiarity with the test (e.g., getting 
acquainted with instructions, item types 
and format, time limits, and tools 
permitted, such as a calculator); 

(2) reviewing content relevant to a test (e.g., 
reviewing the content of the mathematical 
reasoning portion of a test); and 

(3) learning "tricks" or testwiseness skills, 
such as strategies for guessing, eliminating 
choices in multiple-choice questions, using 
time efficiently, and avoiding errors 
(M i liman, Bishop, & Ebel, 1965). 

Some programs emphasize the content of tests, 
while others stress quick tricks or testwiseness 
strategies. J ackson (1980) has noted that distinc- 
tions between short-term programs emphasizing 
drill and testwiseness and long-term programs 
that more closely resemble supplementary educa- 
tion should be distinguished when examining 
research on coaching. The College Board has 
asserted that familiarity with the test is impor- 
tant— no student should take the test "cold" — 
and most test takers seem to heed this advice. For 
instance, Powers (1998) reported that more than 
97 percent of students prepared in some way 
before taking the SAT I, with a median of about 11 
hours of preparation. Consistent with its advice, 
the College Board provides free of charge an expla- 
nation of test instructions, timing and test-taking 
strategies, a description of the content and format 
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of items, and a sample test to all students regis- 
tering to take the SAT I. 

In addition to test familiarization activities, 
some test takers undertake much more extensive 
preparation. For instance, Schwartz (1999) 
described an intensive yearlong tutoring program 
that "isn't so much teaching the test as simply 
teaching— in this case, intensive tutorials in math 
or reading" ( p. 56). Reportedly, this program has 
resulted in large score gains on the SAT I. The 
assertions are based largely on anecdotal reports 
instead of formal research. Such programs, which 
extend well over a year in duration, differ from 
more popular test preparation programs, as they 
focus much more heavily on content that is rele- 
vant not only to the test, but also to education and 
learning. Such instruction may come at a very 
high price (fees of $415 per session were reported 
by Schwartz). 

The College Board (1999) has acknowledged 
that such long-term, intensive preparation that 
resembles genuine education may improve 
reasoning skills that are both measured by the 
SAT I and required for success in college. While no 
controlled research study has yet examined the 
effects of such intensive tutoring programs, the 
College Board has maintained that students' math 
and verbal reasoning skills (and their SAT I scores) 
can improve as a result of rigorous academic study 
and other efforts (both in school and out of 
school). In evaluating the effects of coaching 
programs on test scores, the duration of the 
program and the focus of the program should be 
considered. 

Formal, commercial test preparation 
programs tend to emphasize score increases, 
while short-term in-school programs and work- 
shops appear to focus primarily on familiarity 
with the test. Increasing student confidence and 
reducing anxiety are also common across most 
forms of test preparation. 

HOWTHE RIGORAND DESIGN OF 
COACHING STUDIES AFFECT 
THE RESULTS 

In 1990, Becker identified nearly 50 studies of the 
effects of coaching for the SAT, and nearly a dozen 
additional studies have been conducted sincethat 
time. Studies of coaching differ in a number of 


important areas, making results across studies 
difficult to compare. Many of the differences found 
in studies examining the effects of coaching on the 
SAT "relate less to the nature of the coaching inter- 
vention and more to the (quality) of the coaching 
research itself" (Becker, 1990, p. 376). Examining 
methodological differences among coaching 
studies can help to explain their discrepant 
results. 

• Uncontrolled studies examine the score 
changes only for students who have attended 
test preparation programs. No effort is made 
to examine score gains for comparable 
students not attending coaching programs. 
DerSimonian and Laird (1983) noted that 
mean gains from such studies are about 40 
for SAT Verbal and 50 points for Math scores. 
In such studies, any test score gain, even 
from a test taken up to two years earlier, is 
assumed to result solely from the coaching 
programs. Nearly all reports from commer- 
cial and formal test preparation programs are 
based on such simple score gains of students 
completing their programs. Sometimes the 
selective reporting of results also seems to 
be a factor. Several researchers have 
attempted to improve upon such studies by 
subtracting the typical gain for all repeat test 
takers (e.g., Slack & Porter, 1980). Flowever, 
this strategy does not adequately control for 
differences between coached and uncoached 
test takers. These uncontrolled differences, 
and not the results of being coached, may 
explain any apparent effects of coaching. 
Such studies represent the least rigorous 
evaluations of the effects of coaching. 

• At a minimum, controlled studies examine 
score gains for a second group of students 
who did not enroll in formal coaching 
programs. The effect of coaching is defined 
as the difference between the improvement 
made by coached and uncoached (control) 
groups. For more precise estimates, some 
studies employ matched or randomized 
groups in a further attempt to minimize any 
differences (e.g., grades, prior test scores, 
ethnicity, family income, and education) 
between students in the coached and 
uncoached groups, thereby rendering the 
groups as comparable as possible. Such 
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TABLE 1 

MEAN COACHING EFFECT ONTHE SAT BY QUALITY OF COACHING STUDY 

Study design 

Mean (standard error) effect 
on SAT Verbal 

Mean (standard error) effect 
on SAT Math 

U nco ntrolled, single- group studies 

40.6 (10.1) 

53.8 (2.6) 

Controlled studies 

15.3 (5.5) 

15.6 (2.6) 

Randomized or matched groups 

10.1 (3.5) 

9.8 (3.8) 


studies are considered to be the most 
rigorous and objective evaluation of 
coaching's potential effects (Becker, 1990; 
Powers, 1993). 

• Meta-analysis is an analytic technique that 
enables researchers to examine the effects 
across a large number of individual studies 
that are often noncomparable due to differ- 
ences in designs, samples, and conditions. 
This technique permits integration or 
synthesis to examine noncomparable studies 
and provides the most powerful estimate of 
the effects of coaching. Four such meta- 
analyses have been conducted on SAT test 
preparation programs. 

Several consistent findings emerge from previous 
research. First, the effects of coaching programs 
are four to five times greater in studies that do not 
include a control group than in controlled scien- 
tific studies. Results from DerSimonian and Laird 
(1983) are illustrated in Table 1 and demonstrate 
that more rigorous and controlled studies show 
substantially smaller effects from coaching. The 
use of a control group is a standard practice in 
scientific studies in fields as diverse as medicine, 
finance, and education. 

The importance of employing a control or 
comparison group can be easily illustrated. For 
example, an unpublished study at Deerfield 
Academy in Massachusetts (Franker, 1986-87) 
found that 19 students completing a commercial 
test preparation course increased their total SAT 
scores by an average of 80 points. This "effect" of 
coaching was identical to the average gain for 119 
uncoached students at Deerfield. If a comparison 
group had not been used in this study, the 
increase of 80 points would have appeared quite 
impressive. When the score increases for students 
not attending formal test preparation courses are 
also considered, the effects of test preparation are 


usually reduced dramatically. 

To date, commercial coaching firms have 
documented their claims of large score increases 
mostly through uncontrolled single-group surveys 
of students completing their programs. Although 
sometimes verified by accounting firms, such 
surveys do not include control groups, have not 
been published in peer-reviewed journals, and do 
not constitute credible scientific studies (Powers 
and Rock, 1998). In many of these studies, the 
pre-test is not even an actual operational form of 
the SAT I or PSAT/NMSQT, but rather either a 
"facsimile" or a retired form of the SAT I adminis- 
tered under nonoperational conditions. Esti- 
mating score gain from an initial test that is not 
operational is a major flaw because students are 
less likely to be motivated and test-taking condi- 
tions are likely to be substantially different from 
operational administrations. Less effort and moti- 
vation are likely to result in much lower initial 
scores than an actual test administration and 
therefore inaccurately high estimates of score 
increases. 

TEST PREPARATION TODAY 

Powers (1998) reports that in 1995-96, students 
spent an average of 11 hours in preparation for the 
SAT I, an increase of one hour in the past nine years. 
At the extremes, 3 percent devoted no time to test 
preparation, and about half of the students spent no 
money on it, while 10 percent spent at least 54 
hours and over $135. The 12 percent of students 
attending external coaching programs spent the 
most money on test preparation activities. 

Taking the PSAT/NMSQT was the most 
frequent form of test preparation (81 percent), 
with reading and completing the sample test in the 
free booklet provided by the College Board, 
Taking the SAT I, cited by over half of students 
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preparing for the SAT I. About 40 percent of 
students reviewed math books and 33 percent 
reviewed English books or vocabulary in prepara- 
tion for the test, and about one third of all respon- 
dents received preparation for the SAT I as part of 
their regular classroom instruction. Similarly, one 
third of students purchased test preparation 
books and nearly 20 percent purchased software. 
While 12 percent of students attended coaching 
programs in 1995-96, 10 percent of students did so in 
1992 and 11 percent in 1986-87. Similarly, between 5 
percent and 7 percent of students reported 
attending private tutoring in surveys completed 
between 1986 and 1996. The only significant depar- 
tures appear to be an increase in the use of test 
preparation software in recent years and a 
decrease in the number of students reviewing 
English books or vocabulary on their own. 

STUDYING COACHING ONTHE 
NEW SAT I 

Powers and Rock (1998) conducted the first study 
of the effects of formal and out-of-school coaching 
programs on the new SAT I. They surveyed a strati- 
fied random sample of juniors and seniors who had 
registered for the SAT I in 1995-96. The approxi- 
mately 4,200 students who responded to the 
surveys (63 percent of the sample) were represen- 
tative of the larger college-bound population of test 
takers (Powers, 1998). A total of 507 students (12 
percent) reported attending an out-of-school 
coaching program, with 220 students attending a 
program conducted by one of the two major 
national test preparation firms, with an additional 
287 students attending programs conducted by 
other firms or organizations. 

Only those students who first completed an 
actual administration of the PSAT/NMSQT or SAT I 
prior to attending the formal external coaching 
program and then taking (or retaking) the SAT I 
were included in the "coaching group." 1 Similarly, 
the students in the uncoached group, who served 
as the control group, also completed a 
PSAT/NMSQT or SAT I during the same period of 
time and then took (or retook) the SAT I. This 
constituted a controlled study, and it was the 
difference between the score gains for each group 


that best represented the additional boost or gain 
associated with coaching. 

COACHED STUDENTS DIFFER 
FROM OTHER STUDENTS IN A 
NUMBER OF WAYS 

Students who enroll in formal external coaching 
programs tend to differ from uncoached students 
in a number of ways, several of which are also 
related to performance on the SAT I. Table 2 
reveals that coached students are more likely to: 

• come from families with more formal educa- 
tion and higher income 

• report their ethnicity as Asian American 

• have higher degree aspirations 

• have taken slightly more high school courses 
in math, science, and foreign language 

• aspire to more competitive colleges (higher 
mean SAT I scores among applicants) 

Coached students are also more likely to engage in 
multiple test preparation strategies than 
uncoached students. Again, Table 2 illustrates that 
twice as many coached students purchased the 
College Board's Real SATs or other test prepara- 
tion books, received private tutoring, and used 
study aids in preparation for the SAT I. Coached 
students were also more likely to use test prepara- 
tion software, receive some SAT I preparation in 
high school classes, and have taken the tests 
(PSAT/NMSQT and SAT I) before. Overall, students 
attending external coaching programs were more 
likely to engage in nearly all forms of test prepara- 
tion, some at twice the rate, than students who 
did not enroll in such formal programs. Coached 
students were also much less likely to regard their 
earlier test scores as good estimates of their abili- 
ties, and they indicated that good scores on the 
SAT I were of more importance to them than 
uncoached students did. 

The substantial differences in many of these 
descriptive and demographic categories between 
the groups of coached and uncoached students 
were a concern to Powers and Rock (1998), as 
some of these characteristics could potentially 
affect estimates of the effects of coaching. 
Students who did not enroll in external coaching 
programs prepared for the test in different ways. 


^hat is, to examine the effects of external coaching programs, students first must have taken the PSAT/N MSQT or SAT I (which could provide baseline 
data on their performance) prior to coaching, and subsequently take the SAT I following coaching. 
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TABLE 2 

PERCENTAGE OF COACHED AND UNCOACHED STUDENTSAND DIFFERENCES IN 
DEMOGRAPHIC AND BACKGROUND CHARACTERISTICS* 

Characteristic 

Coached 

Students 

Uncoached 

Students 

x 2 

Female 

Ethnicity 

59 

59 

0.0 

89.7** 

Asian American 

21 

8 


African American 

11 

9 


Hispanic 

6 

8 


W hite 

58 

72 


Other Minority 

5 

4 


English Best Language 
Father's Education 

95 

98 

18.2** 

116.2** 

H igh school or less 

13 

26 


Some college, associate's or bachelor's degree 

40 

49 


Some graduate school or degree 

Mother's Education 

47 

25 

89.6** 

High school or less 

18 

31 


Some college, associate's or bachelor's degree 

49 

51 


Some graduate school or degree 

Parents’ Income 

34 

18 

156.0** 

Less than $40,000 

23 

39 


$40,000-$80,000 

34 

43 


Above $80,000 

High School Grade-Point Average 

43 

18 

25.2** 

A average 

46 

41 


B average 

45 

48 


C average 

Degree Goal 

9 

11 

36.2** 

Bachelor's degree or less 

15 

25 


Master's or higher 

69 

55 


Undecided 

Test Preparation 

16 

20 


Read Taking the SAT 1 

57 

58 

0.4 

Tried sample test 

51 

51 

0.1 

Got College Board's Real SATs 

21 

9 

81.2** 

Got College Board's Intro. New SAT 1 

9 

6 

11.7** 

Got College Board's video, Inside the SAT 1 

2 

2 

0.3 

Got other test prep books 

62 

28 

239.9** 

Got special SAT 1 prep in high school class 

39 

32 

9.3** 

Attended special prep given by school 

19 

18 

0.5 

Tutored privately 

15 

5 

75.0** 

Used test prep software 

26 

18 

18.8** 

Used study aids 

49 

21 

186.5** 

Accessed test prep online 

2 

1 

1.8 

Used videos or related resources 

2 

2 

0.0 

Reviewed materials from math courses on own 

35 

39 

3.6 

Reviewed materials from English courses on own 

30 

33 

1.9 

Previously took PSAT /N M SQ T 

88 

80 

18.3** 

Previously took SAT 1 

73 

54 

73.2** 

0 ther 

Perception of Previous SAT 1 or PSAT Score 

16 

12 

7.3** 

45.7** 

Pretty good estimate of my abilities 

20 

32 


Somewhat too low compared with my abilities 

54 

50 


Much too low compared with my abilities 

NervousnessTaking the Most Recent SAT 1 

26 

18 

34.9** 

Extremely or Very N ervous 

35 

23 


Somewhat N ervous 

31 

29 


Slightly N ervous or N ot N ervous 

Importance of Good SAT Scores 

33 

47 

29.8** 

Extremely Important 

52 

40 


Very Important 

37 

41 


Somewhat Important 

10 

17 


Slightly Important or N ot Important 

1 

3 


N ote: For coached examinees, ns ranged from 397 to 534 for questions. For uncoached examinees, comparable ns ranged from 2,782 to 3,733. 
* Adapted from Powers and Rock (1998). 

** p < .01. 
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However, one concern was that students who did 
not enroll in formal external coaching programs 
would be more likely to attend school-based test 
preparation classes or workshops, receive prepa- 
ration in school classes, and use private tutors, 
software, or test preparation books. This would 
make comparisons between groups and estimates 
of score increases more difficult. On the contrary, 
Table 2 illustrates that coached students, on 
average, engaged in other test preparation activi- 
ties more frequently than did uncoached students. 
Therefore, when comparing raw score gains on the 
SAT I between both groups, any residual gain for 
coached students could also be partly explained 
by their propensity to devote more time to other 
test preparation activities, such as software, 
tutors, and test preparation books, as opposed to 
attributing the total residual gain to coaching 
alone. 

While randomized groups of students were 
not used in this study, several different analytic 
methods, including methods to match comparable 
students, were used to estimate the effects of 
coaching on SAT I scores. Each of the methods 
controlled in slightly different ways for preexisting 
differences between coached and uncoached test 
takers. These analytic procedures are generally 
preferable and more feasible for coaching studies 
than are randomized groups. 

A simple comparison between the score 
gains for the groups was also made to establish a 
baseline. Other methods attempted to control for 
differences between groups (e.g., parental educa- 


tion and income, degree aspirations, and earlier 
test scores) that could also affect the latter SAT I 
scores for groups. Still other methods attempted 
to match cases or control for selection bias 
between groups in different ways. More detailed 
explanations of these analyses and their rationale 
are provided in Powers and Rock (1998). Here, we 
summarize results across these analyses, placing 
greater emphasis on the simpler models. 2 

EFFECTS OF COACHING ONTHE 
SAT I 

Table 3 illustrates the raw score differences 
between groups attending external coaching 
programs and other students. Coached students 
showed a 29-point gain on the verbal score, as 
opposed to a 21-point gain for uncoached 
students. In math, the gains are 40 points and 22 
points, respectively, for coached and uncoached 
groups. As discussed earlier, the effect of a 
coaching program on SAT I score increases is not 
simply the average increase between admission 
tests completed prior to and then subsequent to a 
coaching program. Computing such score gains, 
as often done by coaching firms, is flawed 
because it doesn't consider the average growth 
for comparable students attributable to student 
development, maturation, educational experi- 
ences both in and out of school , increased famil- 
iarity with the test, and error. However, when 
examining score gains for coached students 
alone, the 29- and 40-point gains are much smaller 


TABLE 3 

MEAN PRE-SAT I, POST-SAT I, AND GAIN SCORES 
(AND STANDARD DEVIATIONS) FORCOACHED AND UNCOACHED EXAMINEES* 


Group 

Pre-SAT 1 

POst-SAT 1 

Gain 

Coaching Effect 

Verbal 

Coached (n=427) 

500 (92) 

529 (97) 

29 (59) 

29- 21 =8 

Uncoached (n= 2,733) 

506 (101) 

527 (101) 

21 (52) 

- 

Math 

Coached (n=427) 

521 (100) 

561 (100) 

40 (58) 

40 - 22 = 18 

U ncoached (n=2,733) 

505 (101) 

527 (101) 

22 (50) 

- 


* Adapted from Powers and Rock (1998). 


2 The Repeated Measures design only adjusts for differences in initial test scores between groups, and the Analysis of Covariance statistically controls for 
between-group differences on nine variables, including initial test scores. 
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than claims by national test preparation firms. 

When the score gains for coached and 
uncoached groups are compared, the estimated 
effect of coaching is 8 points on verbal and 18 
points on math. This residual gain (subtracting 
the gain for one group from the gain for the 
second group) does not control for the differences 
between groups in various characteristics shown in 
Table 2, which can also affect score gain. 

The effects of coaching were estimated 
based on raw score differences, as noted above, 
as well as with six additional statistical models. 
Table 4 illustrates the effects of raw scores, 
repeated measures, and analysis of covariance 
designs (see Powers and Rock, 1998, for full expla- 
nation and estimates for other statistical models). 
The effect of coaching is fairly consistent across all 
of the analysis models, with a median effect of 8 
points for verbal scores and 18 points for math. 

The various analysis models were expected 
to yield somewhat different results, yet the 
largest average effect for coaching found using 
any of these methods was 34 total points across 
both verbal and math scores. We can conclude 
that the typical effect of coaching, whether aver- 
aging across various models of analysis, using 
simple unadjusted raw score changes, or 
employing some adjustments as done in repeated 
measures or ANCOVA, is 8 points on SAT I Verbal, 
18 points on SAT I Math, and a total of 26 points 
across both scores. 

In addition, this study concluded: 

• Coached students were slightly more likely to 
experience large score increases than were 
uncoached students. Twelve and 16 percent 
of coached students had increases of 100 
points or more on verbal and math scores, 
respectively, compared with 8 percent for 
uncoached students (same percentage for 


both math and verbal scores). 

• About one third of all students actually had 
no gain or a loss when retesting. On the 
verbal scale, 36 percent of coached students 
and 37 percent of uncoached students had a 
score decrease or no increase when 
retesting. On the math scale, 28 percent of 
coached students had a decrease or no 
increase compared to 37 percent of 
uncoached students. 

• Students attending one of the two largest 
coaching firms' preparation programs do fare 
better than students attending other external 
coaching programs, but again, the effects of 
coaching are still small. The typical gains for 
students attending these firms were 14 and 8 
points on verbal scores and 11 and 34 points 
on math scores (with an average increase of 
10 points on verbal, 22 points on math, and 
43 points on combined verbal plus math for 
the two major test preparation firms). 

•There are no significant differences in the 
effects of coaching on the basis of gender 
and race or ethnicity, and whether initial 
scores were high or low. Students with 
initially lower SAT I scores are more likely to 
experience large increases than were 
students with initially higher SAT I scores for 
statistical reasons. However, the effects of 
coaching remain constant across students at 
all initial score levels after controlling for the 
typical score gains achieved by students. 

• The revised SAT I is no more coachable than 
the previous SAT. 

DISCUSSION AND CONCLUSIONS 

Results from Powers and Rock (1998) are quite 
consistent with nearly all published research on 


TABLE 4 

EFFECTS OF COACHING BASED ON ALTERNATIVE MODELS OF ANALYSIS* 

Analysis Model 

SAT 1 Verbal 

Mean Effect / Standard Error 

SAT 1 Math 

Mean Effect / Standard Error 

Raw Score (n = 427) 

8 (3) 

18 (3) 

Repeated Measures (n = 427) 

8 (3) 

18 (3) 

ANCOVA (n = 235) 

6 (4) 

18 (3) 

♦Adapted from Powers and Rock (1998). 
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the effects of coaching and especially consistent 
with results from meta-analyses and controlled 
studies. A summary of findings from the four meta- 
analyses of the SAT estimated similar mean effects 
for coaching: 

• In the most recent and comprehensive meta- 
analysis, Becker (1990) estimated a gain of 9 
points on the SAT Verbal score and 19 points 
on the SAT Math scale, using studies with 
comparison groups. 

• Kulik, Bangert-Drowns, and Kulik (1984) 
examined only coaching programs that expli- 
cated instructed test taking strategies (rather 
than those focused primarily on teaching 
content) and reported coaching effects of 
approximately 15 to 16 points for verbal and 
math sections of the SAT. 

• Results from DerSimonian and Laird (1983) 
are reported in Table 1. Using studies 
employing a comparison group, average 
effects for coaching are 15 points on verbal 
and math scores. 

• The study by Messick and J ungeblut (1981) 
estimated a mean coaching effect of 14 points 
on SAT Verbal scores and 15 points on SAT 
Math scores using controlled studies. 

The exact effect of test preparation for the SAT 
and SAT I cannot be stated unequivocally. 
However, results across four meta-analyses, over 
two dozen controlled studies, and the most recent 
controlled study by Powers and Rock (1998) 
suggest that formal test preparation programs are 
likely to result in an increase of between 9 and 15 
points on the verbal scale and 15 and 18 points on 
the math scale. Larger score gains are usually 
found in math than in verbal. These figures are 
increases attributed to coaching that are beyond 
the average increase found across groups of 
students who did not attend formal coaching 
programs. 

A number of other additional conclusions 
can be inferred from the body of research exam- 
ining the effects of coaching on the SAT and SAT I: 

• The quality of the study, whether a single 
pre- or post-test gain is computed for 
students receiving test preparation, or 
whether more scientifically rigorous 
methods are used, will have a substantial 
impact on the findings. The more rigorous 
the study, the smaller the effects; and exam- 


ining simple score gain for coached students 
in a pre- or post-test design can be quite 
misleading. 

• Longer coaching programs result in slightly 
larger score gains than shorter programs. 
After approximately three to nine hours, 
additional time devoted to formal test prepa- 
ration appears to have diminishing returns. 

• We are aware of no controlled research that 
has examined the effects of intensive and 
long-term tutoring or supplementary educa- 
tional programs geared toward improving 
reasoning skills and content expertise. 
However, previous research has demon- 
strated that rigorous coursework and study 
can improve reasoning skills and perfor- 
mance on the test. A logical extension of this 
work would suggest that such long-term 
interventions, which more closely resemble 
rigorous and long-term academic study than 
commercial coaching programs that focus on 
tricks, strategies, or testwiseness, may 
increase students' academic skills and 
overall performance on tests that measure 
these skills. 

• Students who enroll in formal coaching 
programs for the SAT I differ from other 
students in terms of their aspirations, satis- 
faction with their previous test scores, family 
background and income, and ethnicity. Such 
differences make simple comparisons between 
groups flawed. 

•The effects of coaching on SAT scores are 
considerably smaller than the effects of 
coaching for other aptitude and achievement 
tests (research summarized in Powers, 1993). 

SUMMARY 

The relationship between coaching and subse- 
quent performance on the SAT I is a complex 
issue. Overall, anecdotal reports from one or two 
students and even studies that gauge the effects of 
coaching by examining the score gains for 
students who have enrolled in these programs are 
highly misleading and will not present an accurate 
picture of the true effects of coaching. 

Collectively, these studies indicate that 
coaching does have some benefit. However, 
though consistent, the effect is small. A typical 
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student enrolling in a formal test preparation 
program can expect a total score gain of between 
25 to 32 total points on the verbal and math 
sections beyond what they could expect to gain 
without attending such a program. The largest 
effect found across the meta-analyses or the 
various analytic methods employed in the most 
recent controlled study (Powers and Rock, 1998) 
would amount to about two to four more items 
correct on the 60-item math section and one to 
two additional items correct on the 78-item verbal 
section. Assuming that a typical external coaching 
program may cost over $750 and require up to 40 
hours of classroom instruction (and perhaps 
another 10 to 20 hours of homework assignments), 
students and parents should weigh carefully the 
decision to attend a coaching program against the 
benefits of devoting that time to other education- 
ally relevant activities and extracurricular activi- 
ties that are viewed as equally important for 
admission to, as well as eventual success in, 
college. A rigorous program of high school 
courses, along with familiarity with the content 
and format of the SAT I and practice on actual 
tests, can help students prepare for college and 
the SAT I. 

The authors are Donald E. Powers, a principal 
research scientist at Educational Testing Service, 
and Wayne J. Camara, executive director of 
research and development at the College Board. 
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